Perceptual Intelligence in Cortana Analytics

Exploring the Microsoft Project Oxford Face API

In this activity, you will use Python to explore various methods available from the Face API. You will detect a face in an image and crop the image so only the face appears. You will then detect multiple faces in a second image and similarly crop each face. After getting exposure to detecting faces, you will then create a face list and add five distinct faces to the list. Finally, you will use your initial image to find similar faces from your face list.

This activity uses the following API methods:

* Detect
* Create a Face List
* Get a Face List
* Delete a Face List (optional)
* Add a Face to a Face List
* Find Similar

All images are in the public domain.

Step 1: Enter your Face API key

Substitute the value of faceApiSubscriptionKey with your own API key. Replace [Face API Primary Key] but leave the quotation marks around your key.

To obtain your Face API key or sign up for the API, visit the Subscription page.

When ready, run the cell, which imports required libraries and sets initial variables.



In [ ]:

    
import httplib, urllib, base64, json
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np
%matplotlib inline

faceApiSubscriptionKey = "[Face API Primary Key]"

baseFaceUrl = "https://raw.githubusercontent.com/CortanaAnalyticsLabs/\
CortanaAnalyticsLabs/master/PerceptualIntelligence/Images/"

thumbsUp = baseFaceUrl + "ThumbsUp.png"
multipleFaces = baseFaceUrl + "MultipleFaces.png"
faceA = baseFaceUrl + "FaceA.png"
faceB = baseFaceUrl + "FaceB.png"
faceC = baseFaceUrl + "FaceC.png"
faceD = baseFaceUrl + "FaceD.png"
faceE = baseFaceUrl + "FaceE.png"

print("Initialization complete")

Step 2: Define functions

The following five functions allow you to pass your API key and other relevant parameters based on the API reference. Run the cell to create the various functions.

* detect(apiKey, imageUrl)
* createFaceList(apiKey, faceListId)
* getFaceList(apiKey, faceListId)
* deleteFaceList(apiKey, faceListId)
* addFaceToFaceList(apiKey, faceListId, imageUrl)
* findSimilars(apiKey, faceListId, faceId, numberOfCandidates)



In [ ]:

    
def detect(apiKey, imageUrl):
    jsonBody = '{"url":"' + imageUrl + '"}'

    headers = {
        'Content-Type': 'application/json',
        'Ocp-Apim-Subscription-Key': apiKey,
    }

    params = urllib.urlencode({
        'returnFaceId': 'true',
        'returnFaceLandmarks': 'false',
        'returnFaceAttributes': 'age,gender',
    })

    try:
        conn = httplib.HTTPSConnection('api.projectoxford.ai')
        conn.request("POST", "/face/v1.0/detect?%s" % params, jsonBody, headers)
        response = conn.getresponse()
        data = response.read()
        print(data)
        conn.close()
    except Exception as e:
        print("[Errno {0}] {1}".format(e.errno, e.strerror))
        
    return data

def createFaceList(apiKey, faceListId):
    faceList = faceListId
    jsonBody = '{"name":"' + faceList + '", "userData":"User-provided data attached to the face list"}'
    
    headers = {
        'Content-Type': 'application/json',
        'Ocp-Apim-Subscription-Key': apiKey,
    }

    params = urllib.urlencode({
    })

    try:
        conn = httplib.HTTPSConnection('api.projectoxford.ai')
        conn.request("PUT", "/face/v1.0/facelists/" + faceList + "?%s" % params, jsonBody, headers)
        response = conn.getresponse()
        data = response.read()
        print(data)
        conn.close()
    except Exception as e:
        print("[Errno {0}] {1}".format(e.errno, e.strerror))
        
def getFaceList(apiKey, faceListId):
    faceList = faceListId
    jsonBody = ''
    
    headers = {
        'Ocp-Apim-Subscription-Key': apiKey,
    }

    params = urllib.urlencode({
    })

    try:
        conn = httplib.HTTPSConnection('api.projectoxford.ai')
        conn.request("GET", "/face/v1.0/facelists/" + faceList + "?%s" % params, jsonBody, headers)
        response = conn.getresponse()
        data = response.read()
        print(data)
        conn.close()
    except Exception as e:
        print("[Errno {0}] {1}".format(e.errno, e.strerror))
        
def deleteFaceList(apiKey, faceListId):
    faceList = faceListId
    jsonBody = ''
    
    headers = {
        'Ocp-Apim-Subscription-Key': apiKey,
    }

    params = urllib.urlencode({
    })

    try:
        conn = httplib.HTTPSConnection('api.projectoxford.ai')
        conn.request("DELETE", "/face/v1.0/facelists/" + faceList + "%s" % params, jsonBody, headers)
        response = conn.getresponse()
        data = response.read()
        print(data)
        conn.close()
    except Exception as e:
        print("[Errno {0}] {1}".format(e.errno, e.strerror))
        
def addFaceToFaceList(apiKey, faceListId, imageUrl):
    faceList = faceListId
    jsonBody = '{"url":"' + imageUrl + '"}'
    
    headers = {
        'Content-Type': 'application/json',
        'Ocp-Apim-Subscription-Key': apiKey,
    }

    params = urllib.urlencode({
    })

    try:
        conn = httplib.HTTPSConnection('api.projectoxford.ai')
        conn.request("POST", "/face/v1.0/facelists/" + faceList + "/persistedFaces?%s" % params, jsonBody, headers)
        response = conn.getresponse()
        data = response.read()
        print(data)
        conn.close()
    except Exception as e:
        print("[Errno {0}] {1}".format(e.errno, e.strerror))
        
def findSimilars(apiKey, faceListId, faceId, numberOfCandidates):
    jsonBody = '{"faceId":"' + faceId + '", \
                "faceListId":"' + faceListId + '", \
                "maxNumOfCandidatesReturned":' + numberOfCandidates + '}'
    headers = {
        'Content-Type': 'application/json',
        'Ocp-Apim-Subscription-Key': apiKey,
    }

    params = urllib.urlencode({
    })

    try:
        conn = httplib.HTTPSConnection('api.projectoxford.ai')
        conn.request("POST", "/face/v1.0/findsimilars?%s" % params, jsonBody, headers)
        response = conn.getresponse()
        data = response.read()
        print(data)
        conn.close()
    except Exception as e:
        print("[Errno {0}] {1}".format(e.errno, e.strerror))
        
print("API functions created")

Step 3: Display the initial image

Run the following cell to display the image of a person. The image also displays pixels along the two axes.



In [ ]:

    
img = mpimg.imread(thumbsUp)
plt.imshow(img)

Step 4: Detect the face on the initial image

Run the following cell to call the detect method, which returns a face box for the image. Note the values of "top" and "left" in "faceRectangle" and compare that point to the axes on the original image. The face rectangle then defines a box using the appropriate "width" and "height" values starting from that point. In addition, gender and age are displayed as attributes.



In [ ]:

    
thumbsUpData = detect(faceApiSubscriptionKey, thumbsUp)

Step 5: Crop the face

Run the following cell to take the JSON returned by the API and display only the faceRectangle



In [ ]:

    
face = json.loads(thumbsUpData)
thumbsUpFaceId = face[0]["faceId"]
faceTop = face[0]["faceRectangle"]["top"]
faceLeft = face[0]["faceRectangle"]["left"]
faceWidth = face[0]["faceRectangle"]["width"]
faceHeight = face[0]["faceRectangle"]["height"]

img = mpimg.imread(thumbsUp)
plt.imshow(img[faceTop:faceTop + faceHeight, faceLeft:faceLeft + faceWidth])

Step 6: Detect multiple faces in an image

Run the following cells in sequence to display an image with multiple faces, call the API to detect faces, and display the results. Note that while there are three people in the image, the API only returns two faces due to the side-facing orientation of the third person.



In [ ]:

    
img = mpimg.imread(multipleFaces)
plt.imshow(img)



In [ ]:

    
multipleFacesData = detect(faceApiSubscriptionKey, multipleFaces)



In [ ]:

    
face = json.loads(multipleFacesData)
faceTop = face[0]["faceRectangle"]["top"]
faceLeft = face[0]["faceRectangle"]["left"]
faceWidth = face[0]["faceRectangle"]["width"]
faceHeight = face[0]["faceRectangle"]["height"]

img = mpimg.imread(multipleFaces)
plt.imshow(img[faceTop:faceTop + faceHeight, faceLeft:faceLeft + faceWidth])



In [ ]:

    
face = json.loads(multipleFacesData)
faceTop = face[1]["faceRectangle"]["top"]
faceLeft = face[1]["faceRectangle"]["left"]
faceWidth = face[1]["faceRectangle"]["width"]
faceHeight = face[1]["faceRectangle"]["height"]

img = mpimg.imread(multipleFaces)
plt.imshow(img[faceTop:faceTop + faceHeight, faceLeft:faceLeft + faceWidth])

Step 7: Create a Face List

Run the following cell to create a new face list called "sample", then show that the list is empty. A Face List is simply a collection of faces that remain unidentified and are referenced by Id. The API also has the ability to create a Person List for known people and call methods to identify new face images based on that list. Note that there is also a delete function that is commented out in case you would like to delete and re-create your face list at a later time.



In [ ]:

    
#deleteFaceList(faceApiSubscriptionKey, "sample")
createFaceList(faceApiSubscriptionKey, "sample")
getFaceList(faceApiSubscriptionKey, "sample")

Step 8: Add faces to your Face List

Run the following five cells in sequence to display images of five distinct people and load their face information into your face list. Your faces are referenced in the list using "persistedFaceId".



In [ ]:

    
img = mpimg.imread(faceA)
plt.imshow(img)
rawFaceDataA = detect(faceApiSubscriptionKey, faceA)
addFaceToFaceList(faceApiSubscriptionKey, "sample", faceA)



In [ ]:

    
img = mpimg.imread(faceB)
plt.imshow(img)
rawFaceDataB = detect(faceApiSubscriptionKey, faceB)
addFaceToFaceList(faceApiSubscriptionKey, "sample", faceB)



In [ ]:

    
img = mpimg.imread(faceC)
plt.imshow(img)
rawFaceDataC = detect(faceApiSubscriptionKey, faceC)
faceDataC = json.loads(rawFaceDataC)
addFaceToFaceList(faceApiSubscriptionKey, "sample", faceC)



In [ ]:

    
img = mpimg.imread(faceD)
plt.imshow(img)
rawFaceDataD = detect(faceApiSubscriptionKey, faceD)
addFaceToFaceList(faceApiSubscriptionKey, "sample", faceD)



In [ ]:

    
img = mpimg.imread(faceE)
plt.imshow(img)
rawFaceDataE = detect(faceApiSubscriptionKey, faceE)
addFaceToFaceList(faceApiSubscriptionKey, "sample", faceE)

Step 9: Display contents of your Face List

Run the following cell to display the contents of your recently populated face list. When you first created the list and called this function, it was empty. You should now see the "persistedFaceId" values for the five faces that you added to your list.



In [ ]:

    
getFaceList(faceApiSubscriptionKey, "sample")

Step 10: Pass a sample image and find similar faces in your Face List

Run the following cell to compare the initial face of the "thumbs up woman" from Step 3 with the five faces in your face list. One result should display with a reasonably high confidence value. The "thumbs up woman" and "woman with apple" are the same person.



In [ ]:

    
findSimilars(faceApiSubscriptionKey, "sample", thumbsUpFaceId, "5")

Conclusion

You have completed the Face API lab activity. If you would like more detail about additional capabilities, visit the Face API reference page.



In [ ]: